Autonomous Multi-agent Systems Research Articles

Transforming the Testing and Evaluation of Autonomous Multi-Agent Systems: Introducing In-Situ Testing via Distributed Ledger Technology

Transforming the Testing and Evaluation of Autonomous Multi-Agent Systems: Introducing In-Situ Testing via Distributed Ledger Technology

Secure dynamic event-triggering control for consensus under asynchronous denial of service

IntroductionThis article proposes a secure implementation for consensus using a dynamic event-triggered control (DETC) scheme for general autonomous multi-agent systems (MAS) under asynchronous (distributed) denial of service (DoS) attacks. The asynchronous DoS attacks can block each communication channel independently in an unknown pattern. Depending on the impact of DoS on the communication topology, the attacks are categorized into (i): connectivity-preserved DoS (CP-DoS), and (ii): connectivity-broken DoS (CB-DoS). In CP-DoS, the operating communication topology remains connected. On the other hand, in CB-DoS the adversary breaks the communication graph into isolated sub-graphs.MethodsThe DETC scheme is employed to reduce the control updates for each agent. To guarantee consensus under both the CP-DoS and CB-DoS, a linear matrix inequality (LMI) based optimization approach is proposed, which simultaneously designs all the unknown DETC parameters as well as the state feedback control gain.ResultsThe proposed optimization method prioritizes the minimum inter-event interval (MIET) between consecutive control updates. The trade-off between relevant features of the MAS, namely the consensus convergence rate, intensity of control updates, and level of resilience to DoS can be handled by the proposed optimization.DiscussionSimulation results quantify the effectiveness of the proposed approach, showcasing its ability to maintain secure consensus in MAS under varying DoS attack scenarios.

Probability-Based Strategy for a Football Multi-Agent Autonomous Robot System

The strategies of multi-autonomous cooperative robots in a football game can be solved in multiple ways. Still, the most common is the “Skills, Tactics and Plays (STP)” architecture, developed so that robots could easily cooperate based on a group of predefined plays, called the playbook. The development of the new strategy algorithm presented in this paper, used by the RoboCup Middle Size League LAR@MSL team, had a completely different approach from most other teams for multiple reasons. Contrary to the typical STP architecture, this strategy, called the Probability-Based Strategy (PBS), uses only skills and decides the outcome of the tactics and plays in real-time based on the probability of arbitrary values given to the possible actions in each situation. The action probability values also affect the robot’s positioning in a way that optimizes the overall probability of scoring a goal. It uses a centralized decision-making strategy rather than the robot’s self-control. The robot is still fully autonomous in the skills assigned to it and uses a communication system with the main computer to synchronize all robots. Also, calibration or any strategy improvements are independent of the robots themselves. The robots’ performance affects the results but does not interfere with the strategy outcome. Moreover, the strategy outcome depends primarily on the opponent team and the probability calibration for each action. The strategy presented has been fully implemented on the team and tested in multiple scenarios, such as simulators, a controlled environment, against humans in a simulator, and in the RoboCup competition.

An Autonomous Multi-Agent System for Customized Scientific Literature Recommendation: A Tool for Researchers and Students

An Autonomous Multi-Agent System for Customized Scientific Literature Recommendation: A Tool for Researchers and Students

Customization and flexible manufacturing capacity using a graphical method applied on a configurable multi-agent system

• Concurrent use of pluggable resources for quick and agile balance of the manufacturing capacity. • Easy adoption to customized products by utilizing in-house knowledge of the manufacturer. • Graphical tool for easy design and visualization of sequential and parallel goals and requirements. • Plug & Produce robot cell controlled by a configurable multi-agent system verified on a wooden house wall pre-manufacturing case. This article proposes a Plug & Produce and goal-oriented configurable multi-agent system that admits adding and removing resources to balance the manufacturing capacity without doing any digital reconfiguration or reprogramming. To handle that a new part-agent strategy is developed and described. Goals are central in designing autonomous multi-agent systems, possibilities to execute goals in parallel are desirable when the process requirements admit concurrent use of resources. Also, a standardized graphical method, the sequence of goals chart, is proposed to define and visualize parallel and sequential goals independently of available resources. Premanufacturing of wooden houses belongs to one of many manufacturing industries that claim flexible automation systems due to the high degree of customized products and a fluctuating market. A physical Plug & Produce robot-based workstation was built up to verify the flexibility in altering capacity and adoption to product modifications of a house wall section. Further, the simplicity of modifying the proposed configurable multi-agent system was compared to more traditionally designed systems and plain multi-agent systems with superior results. The flexibility is built into the proposed system by default as a part of the concept, simple enough to be handled by existing in-house knowledge within manufacturing companies.

Correctness-guaranteed strategy synthesis and compression for multi-agent autonomous systems

Planning is a critical function of multi-agent autonomous systems, which includes path finding and task scheduling. Exhaustive search-based methods such as model checking and algorithmic game theory can solve simple instances of multi-agent planning. However, these methods suffer from state-space explosion when the number of agents is large. Learning-based methods can alleviate this problem, but lack a guarantee of correctness of the results. In this paper, we introduce MoCReL, a new version of our previously proposed method that combines model checking with reinforcement learning in solving the planning problem. The approach takes advantage of reinforcement learning to synthesize path plans and task schedules for large numbers of autonomous agents, and of model checking to verify the correctness of the synthesized strategies. Further, MoCReL can compress large strategies into smaller ones that have down to 0.05% of the original sizes, while preserving their correctness, which we show in this paper. MoCReL is integrated into a new version of Uppaal Stratego that supports calling external libraries when running learning and verification of timed games models.

Decentralized Multi-Agent Planning Using Model Predictive Control and Time-Aware Safe Corridors

Multi-agent planning has numerous real-world applications, such as autonomous agriculture, search and rescue, and infrastructure inspection. It is thus of tremendous benefit to improve on the state-of-the-art in multiple metrics to make autonomous multi-agent systems more efficient and a more viable real-world solution. In this letter, we propose a new framework for multi-agent planning in a static environment that improves upon the existing state-of-the-art in multiple areas such as computation time, trajectory smoothness, and feasibility. The proposed method first generates a global path that only avoids static obstacles and then generates a Safe Corridor around it. It then extends the notion of Safe Corridors and makes them time-aware in order to account for the future positions of other agents. The time-aware Safe Corridor, along with a local reference trajectory generated from the global path, are used in an Mixed-Integer Quadratic Program/Model Predictive Control formulation that generates a collision-free optimal trajectory. The proposed framework is real-time, decentralized, and synchronous. It is compared to two recent state-of-the-art methods in simulations. It outperforms both methods in robustness as well as feasibility and computation time.

Resilience, reliability, and coordination in autonomous multi-agent systems

Multi-agent systems is an evolving discipline that encompasses many different branches of research. The long-standing Agents at Aberdeen ( A 3 ) group undertakes research across several areas of multi-agent systems, focusing in particular on aspects related to resilience, reliability, and coordination. In this article we introduce the group and highlight past research successes in those themes, building a picture of the strengths within the group. We close the paper outlining the future direction of the group and identify key open challenges and our vision towards solving them.

A Concept of Autonomous Multi-Agent Navigation System for Unmanned Surface Vessels

The paper introduces a proposal of an Autonomous Navigation System for Unmanned Surface Vessels. The system architecture is presented with a special emphasis on collision avoidance and maneuver auto-negotiation. For the purpose of maneuver auto-negotiation, the concept of multi-agent systems has been applied. The algorithm developed for the task of collision avoidance is briefly described and the results of the simulation tests, confirming the effectiveness of applied method, are also given. Presented outcomes include solutions of test scenarios from the perspectives of different ships taking part in the considered situations, confirming the applicability of the collision avoidance algorithm in the process of maneuver auto-negotiation.

Bayesian optimization of distributed neurodynamical controller models for spatial navigation.

Dynamical systems models for controlling multi-agent swarms have demonstrated advances toward resilient, decentralized navigation algorithms. We previously introduced the NeuroSwarms controller, in which agent-based interactions were modeled by analogy to neuronal network interactions, including attractor dynamics and phase synchrony, that have been theorized to operate within hippocampal place-cell circuits in navigating rodents. This complexity precludes linear analyses of stability, controllability, and performance typically used to study conventional swarm models. Further, tuning dynamical controllers by manual or grid-based search is often inadequate due to the complexity of objectives, dimensionality of model parameters, and computational costs of simulation-based sampling. Here, we present a framework for tuning dynamical controller models of autonomous multi-agent systems with Bayesian optimization. Our approach utilizes a task-dependent objective function to train Gaussian process surrogate models to achieve adaptive and efficient exploration of a dynamical controller model’s parameter space. We demonstrate this approach by studying an objective function selecting for NeuroSwarms behaviors that cooperatively localize and capture spatially distributed rewards under time pressure. We generalized task performance across environments by combining scores for simulations in multiple mazes with distinct geometries. To validate search performance, we compared high-dimensional clustering for high- vs. low-likelihood parameter points by visualizing sample trajectories in 2-dimensional embeddings. Our findings show that adaptive, sample-efficient evaluation of the self-organizing behavioral capacities of complex systems, including dynamical swarm controllers, can accelerate the translation of neuroscientific theory to applied domains.

An efficient system using implicit feedback and lifelong learning approach to improve recommendation

This paper presents a new technique for contextual item-to-item Collaborative Filtering-based Recommender System, an improved version popularised by e-commerce giant Amazon two decades back. The concept is based on items also-viewed under the same browsing session. Users’ browsing patterns, locations, and timestamps are considered as the context and latent factors for each user. The algorithm computes recommendations based on users’ implicit endorsements by clicks. The algorithm does not enforce the user to log in to provide recommendations and is capable of providing accurate recommendations for non-logged-in users and with a setting where the system is unaware of users’ preferences and profile data (non-logged-in users). This research takes the cue from human lifelong incremental learning experience applied to machine learning on a large volume of the data pool. First, all historical data is gathered from collectable sources in a distributed manner through big data tools. Then, a long-running batch job creates the initial model and saves it to Hadoop Distributed File System (HDFS). An ever-running streaming job loads the model from HDFS and builds on top of it in an incremental fashion. At the architectural level, this resembles the big data mix processing Lambda Architecture. The recommendation is computed based on a proposed equation for a weighted sum between near real-time and historical batch data. Real-time and batch processing engines act as autonomous Multi-agent systems in collaboration. We propose an ensemble method for batch-stream the recommendation engine. We introduce a novel Lifelong Learning Model for recommendation through Multi-agent Lambda Architecture. The recommender system incrementally updates its model on streaming datasets to improve over time.

Conceptual Microgrid Management Framework Based on Adaptive and Autonomous Multi-Agent Systems

The Smart Grids paradigm emerged as a response to the need to modernize the electric grid and address problems related to the demand for better quality energy. However, there are no fully developed and implemented smart grids, but only some minor scale tests to prove the concepts. Centralized systems are still common, with a low granularity of control and reduced monitoring capacity, especially in low-voltage networks. In this work, we propose a framework for Microgrid Management, addressing problems such as determining how to control the energy demand and peak loads, the effect of the energy consumption in the network, and the amount of energy required. We proposed a solution based on autonomous and distributed systems for the following problems: Peak Load addressed with AIN-DSM distributed algorithm, transformer lifespan estimation using a thermal model adjusted by Genetic Algorithms, and Short-Term Load Forecasting based on Artificial Neural Networks and Genetic Algorithms. The distributed paradigm of the Organization Centered Multi-Agent Systems methodology was applied for the framework's modeling and development. The results obtained by using these solutions in the Tucumán province, Argentina, show the system's capabilities and the relevance of the information produced from the framework.

StigLD: Stigmergic Coordination in Linked Systems

While current Semantic Web technologies are well-suited for data publication and integration, the design and deployment of dynamic, autonomous and long-lived multi-agent systems (MAS) on the Web is still in its infancy. Following the vision of hypermedia MAS and Linked Systems, we propose to use a value-passing fragment of Milner’s Calculus to formally specify the generic hypermedia-driven behaviour of Linked Data agents and the Web as their embedding environment. We are specifically interested in agent coordination mechanisms based on stigmergic principles. When considering transient marker-based stigmergy, we identify the necessity of generating server-side effects during the handling of safe and idempotent agent-initiated resource requests. This design choice is oftentimes contested with an imprecise interpretation of HTTP semantics, or with rejecting environments as first-class abstractions in MAS. Based on our observations, we present a domain model and a SPARQL function library facilitating the design and implementation of stigmergic coordination between Linked Data agents on the Web. We demonstrate the efficacy our of modelling approach in a Make-to-Order fulfilment scenario involving transient stigmergy and negative feedback as well as by solving a problem instance from the (time constrained) Trucks World domain as presented in the fifth International Planning Competition.

E-CNMPC: Edge-Based Centralized Nonlinear Model Predictive Control for Multiagent Robotic Systems

With the wide deployment of autonomous multi-agent robotic systems, control solutions based on centralized algorithms have been developed. Even though these centralized algorithms can optimize the performance of the multi-agent robotic systems, they require a lot of computational effort, and a centralized unit to undertake the whole process. Yet, many robotic platforms like some ground robots and even more, aerial robots, do not have the computing capacity to execute this kind of frameworks on their on onboard computers. While cloud computing has been used as a solution for offloading computationally demanding robotic applications, from the robots to the cloud servers, the latency they introduce to the system has set them unsuitable for time sensitive applications. To overcome these challenges, this article promotes an Edge computing-based Centralized Nonlinear Model Predictive Control (E-CNMPC) framework to control, and optimize, in swarm formation, the trajectory of multiple ground robotic agents, while taking under consideration potential collisions. The data processing procedure for the time critical application of controlling the robots in a centralized manner, is offloaded to the edge machine, thus the framework benefits from the provided edge resources, features, and centralized optimal performance, while the latency remains bounded in desired values. Besides, real experiments were conducted as a proof-of-concept of the proposed framework to evaluate the system’s performance and effectiveness.

Bayesian Optimization of Distributed Neurodynamical Controller Models for Spatial Navigation

Dynamical systems models for controlling multi-agent swarms have demonstrated advances toward resilient, decentralized navigation algorithms. We previously introduced the NeuroSwarms controller, in which agent-based interactions were modeled by analogy to neuronal network interactions, including attractor dynamics and phase synchrony, that have been theorized to operate within hippocampal place-cell circuits in navigating rodents. This complexity precludes linear analyses of stability, controllability, and performance typically used to study conventional swarm models. Further, tuning dynamical controllers by hand or grid search is often inadequate due to the complexity of objectives, dimensionality of model parameters, and computational costs of simulation-based sampling. Here, we present a framework for tuning dynamical controller models of autonomous multi-agent systems based on Bayesian Optimization (BayesOpt). Our approach utilizes a task-dependent objective function to train Gaussian Processes (GPs) as surrogate models to achieve adaptive and efficient exploration of a dynamical controller model's parameter space. We demonstrate this approach by studying an objective function selecting for NeuroSwarms behaviors that cooperatively localize and capture spatially distributed rewards under time pressure. We generalized task performance across environments by combining scores for simulations in distinct geometries. To validate search performance, we compared high-dimensional clustering for high- vs. low-likelihood parameter points by visualizing sample trajectories in Uniform Manifold Approximation and Projection (UMAP) embeddings. Our findings show that adaptive, sample-efficient evaluation of the self-organizing behavioral capacities of complex systems, including dynamical swarm controllers, can accelerate the translation of neuroscientific theory to applied domains.

Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning

Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results extend those in Castellini et al. (Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS’19.International Foundation for Autonomous Agents and Multiagent Systems, pp 1862–1864, 2019) and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements.

Grounded action transformation for sim-to-real reinforcement learning

Reinforcement learning in simulation is a promising alternative to the prohibitive sample cost of reinforcement learning in the physical world. Unfortunately, policies learned in simulation often perform worse than hand-coded policies when applied on the target, physical system. Grounded simulation learning (gsl) is a general framework that promises to address this issue by altering the simulator to better match the real world (Farchy et al. 2013 in Proceedings of the 12th international conference on autonomous agents and multiagent systems (AAMAS)). This article introduces a new algorithm for gsl—Grounded Action Transformation (GAT)—and applies it to learning control policies for a humanoid robot. We evaluate our algorithm in controlled experiments where we show it to allow policies learned in simulation to transfer to the real world. We then apply our algorithm to learning a fast bipedal walk on a humanoid robot and demonstrate a 43.27% improvement in forward walk velocity compared to a state-of-the art hand-coded walk. This striking empirical success notwithstanding, further empirical analysis shows that gat may struggle when the real world has stochastic state transitions. To address this limitation we generalize gat to the stochasticgat (sgat) algorithm and empirically show that sgat leads to successful real world transfer in situations where gat may fail to find a good policy. Our results contribute to a deeper understanding of grounded simulation learning and demonstrate its effectiveness for applying reinforcement learning to learn robot control policies entirely in simulation.

Distributed cooperative control of autonomous multi-agent UAV systems using smooth control

This paper addresses the cooperative control problem of multiple unmanned aerial vehicles (multi-UAV) systems. First, a new distributed consensus algorithm for second-order nonlinear multi-agent systems (MAS) is formulated under the leader-following approach. The algorithm provides smooth input signals to the agents' control channels, which avoids the chattering effect generated by the conventional sliding mode-based control protocols. Second, a new formation control scheme is developed by integrating smooth distributed consensus control protocols into the geometric pattern model to achieve three-dimensional formation tracking. The Lyapunov theory is used to prove the stability and convergence of both distributed consensus and formation controllers. The effectiveness of the proposed algorithms is demonstrated through simulation results.

Demand-Driven Transparency for Monitoring Intelligent Agents

In autonomous multiagent or multirobotic systems, the ability to quickly and accurately respond to threats and uncertainties is important for both mission outcomes and survivability. Such systems are never truly autonomous, often operating as part of a human-agent team. Artificial intelligent agents (IAs) have been proposed as tools to help manage such teams; e.g., proposing potential courses of action to human operators. However, they are often underutilized due to a lack of trust . Designing transparent agents, who can convey at least some information regarding their internal reasoning processes, is considered an effective method of increasing trust. How people interact with such transparency information to gain situation awareness while avoiding information overload is currently an unexplored topic. In this article, we go part way to answering this question, by investigating two forms of transparency: sequential transparency , which requires people to step through the IA's explanation in a fixed order; and demand-driven transparency , which allows people to request information as needed. In an experiment using a multivehicle simulation, our results show that demand-driven interaction improves the operators’ trust in the system while maintaining, and at times improving, performance http://www.ieee.org/documents/taxonomy_v101.pdf .?> and usability.

Agent-Based Distributed Planning and Coordination for Resilient Airport Surface Movement Operations

Airport surface movement operations are complex processes with many types of adverse events which require resilient, safe, and efficient responses. One regularly occurring adverse event is that of runway reconfiguration. Agent-based distributed planning and coordination has shown promising results in controlling operations in complex systems, especially during disturbances. In contrast to the centralised approaches, distributed planning is performed by several agents, which coordinate plans with each other. This research evaluates the contribution of agent-based distributed planning and coordination to the resilience of airport surface movement operations when runway reconfigurations occur. An autonomous Multi-Agent System (MAS) model was created based on the layout and airport surface movement operations of Schiphol Airport in the Netherlands. Within the MAS model, three distributed planning and coordination mechanisms were incorporated, based on the Conflict-Based Search (CBS) Multi-Agent Path Finding (MAPF) algorithm and adaptive highways. MAS simulations were run based on eight days of real-world operational data from Schiphol Airport and the results of the autonomous MAS simulations were compared to the performance of the real-world human operated system. The MAS results show that the distributed planning and coordination mechanisms were effective in contributing to the resilient behaviour of the airport surface movement operations, closely following the real-world behaviour, and sometimes even surpassing it. In particular, the mechanisms were found to contribute to more resilient behaviour than the real-world when considering the taxi time after runway reconfiguration events. Finally, the highway included distributed planning and coordination mechanisms contributed to the most resilient behaviour of the airport surface movement operations.